Adaptive Kernel Methods Using the Balancing Principle

The regularization parameter choice is a fundamental problem in Learning Theory since the performance of most supervised algorithms crucially depends on the choice of one or more of such parameters. In particular a main theoretical issue regards the amount of prior knowledge needed to choose the regularization parameter in order to obtain good learning rates. In this paper we present a parameter choice strategy, called the balancing principle, to choose the regularization parameter without knowledge of the regularity of the target function. Such a choice adaptively achieves the best error rate. Our main result applies to regularization algorithms in reproducing kernel Hilbert space with the square loss, though we also study how a similar principle can be used in other situations. As a straightforward corollary we Communicated by Felipe Cucker. E. De Vito DSA, Università di Genova and INFN, Genova, Italy e-mail: [email protected] S. Pereverzyev Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences, Altenbergerstrasse 69, 4040 Linz, Austria e-mail: [email protected] L. Rosasco ( ) Center for Biological and Computational Learning, Massachusetts Institute of Technology, Cambridge, MA, USA e-mail: [email protected] L. Rosasco DISI, Università di Genova, Genova, Italy 456 Found Comput Math (2010) 10: 455–479 can immediately derive adaptive parameter choices for various kernel methods recently studied. Numerical experiments with the proposed parameter choice rules are also presented.